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Abstract. Maximum flow (and minimum cut) algorithms have had a strong impact on computer 
vision. In particular, graph cuts algorithms provide a mechanism for the discrete optimization of 
an energy functional which has been used in a variety of applications such as image segmentation, 
stereo, image stitching and texture synthesis. Algorithms based on the classical formulation of max- 
flow defined on a graph are known to exhibit metrication artefacts in the solution. Therefore, a 
recent trend has been to instead employ a spatially continuous maximum flow (or the dual min- 
cut problem) in these same applications to produce solutions with no metrication errors. However, 
known fast continuous max-flow algorithms have no stopping criteria or have not been proved to 
converge. In this work, we revisit the continuous max-flow problem and show that the analogous 
discrete formulation is different from the classical max-flow problem. We then apply an appropriate 
combinatorial optimization technique to this combinatorial continuous max-flow (CCMF) problem 
to find a null-divergence solution that exhibits no metrication artefacts and may be solved exactly 
by a fast, efficient algorithm with provable convergence. Finally, by exhibiting the dual problem of 
our CCMF formulation, we clarify the fact, already proved by Nozawa in the continuous setting, 
that the max-flow and the total variation problems are not always equivalent. 

1. Introduction. Optimization methods have been used to address a wide va- 
riety of problems in computer vision. The early optimization approaches were for- 
mulated in terms of active contours and surfaces Q] and then later level sets [2]. 
These formulations were used to optimize energies of varied sophistication (e.g., us- 
ing regional, texture, motion or contour terms [3]) but generally converged to a local 
minimum, generating results that were sensitive to initial conditions and noise levels. 
Consequently, more recent focus has been on energy formulations (and optimization 
algorithms) for which a global optimum can be found. 

Such energy formulations typically include a term which minimizes the boundary 
length (or surface area in 3D) of a region or the total variation of a scalar field in 
addition to a data term and/or hard constraints. In this paper, we focus on image 
segmentation as the example application on which to base our exposition. Indeed, 
segmentation has played a prominent (and early) role in the development of the use 
of global optimization methods in computer vision, and often forms the basis of many 
other applications [H [6] . 

Graph-based approaches. The max-flow/min-cut problem on a graph is a 
classical problem in graph theory, for which the earliest solution algorithm goes back 
to Ford and Fulkerson [TJ. Initial methods for global optimization of the boundary 
length of a region formulated the energy on a graph and relied on max-flow/min-cut 
methods for solution [Sj [9] . It was soon realized that these methods introduced a 
metrication error. Metrication errors are clearly visible when gradient information 
is weak or missing, for example in the case of medical image segmentation or in 
some materials science applications like electron tomography, where weak edges are 
unavoidable features of the imaging modality. Metrication errors are also far more 
obvious in 3D than in 2D and increasingly so, as the dimensionality increases (see 
for example the 3D-lungs example in |10j). Furthermore, metrication artifacts are 
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even more present in other applications such as surface reconstruction and rendering, 
where the artifacts are a lot worse than in image segmentation. Various solutions for 
metrication errors were proposed. One solution involved the use of a highly connected 
lattice with a specialty edge weighting [llj . but the large number of graph edges 
required to implement this solution could cause memory concerns when implemented 
for large 2D or 3D images [10] . 

Continuous approaches. To avoid the metrication artefacts without increas- 
ing memory usage, one trend in recent years has been to pursue spatially continu- 
ous formulations of the boundary length and the related problem of total variation 
[T2l HU1 [T5] . Historically, a continuous max- flow (and dual min-cut problem) was for- 
mulated by Iri [14] and then Strang [15] . Strang's continuous formulation provided 
an example of a continuization (as opposed to discretization) of a classically discrete 
problem, but was not associated to any algorithm. Work by Appleton and Talbot [10] 
provided the first PDE-based algorithm to find Strang's continuous max-flows and 
therefore optimal min-cuts. This same algorithm was also derived by Unger et al. 
from the standpoint of minimizing continuous total variation [16] . Adapted to image 
segmentation, this algorithm is shown to be equivalent [17] to the Appleton- Talbot 
algorithm and has been demonstrated to be fast when implemented on massively par- 
allel architectures. Unfortunately, this algorithm has no stopping criteria and has not 
been proved to converge. Works by Pock et al [IS], Zach et al [H] and Chambolle et 
al [20] present different algorithms for optimizing comparable energies for solving mul- 
tilabel problems, but again, those algorithms are not proved to converge. Some other 
works have been presenting provably converging algorithms, but with relatively slow 
convergence speed. For example, Pock and Chambolle introduce in [21] a general 
saddle point algorithm that may be used in various applications, but needs half an 
hour to segment a 350 x 450 image on a CPU and still more than a dozen seconds on 
a GPU (Details are given in the experiments section). 

Links and differences with total variation minimization. G. Strang [22] 
has shown the continuous max-flow problem for the I2 norm to be the dual of the 
total variation (TV) minimization problem under some assumptions. The problem of 
total variations, introduced successively in computer vision by Shulman and Herve 
[2"5] and later Rudin, Osher and Fatemi [21] as a regularizing criterion for optical 
flow computation and then image denoising, has been shown to be quite efficient for 
smoothing images without affecting contours. Moreover, a major advantage of TV 
is that it is a convex problem. Thus, a straightforward algorithm such as gradient 
descent may be applied to find a minimum solution. However, there is a need for 
faster methods, and significant progress have been achieved recently with primal- 
dual approaches [12], Nesterov's algorithm [35] and Split-Brcgman/Douglas-Rachford 
methods [261 [27] . Most methods minimizing TV focus on image filtering as an ap- 
plication, and even if those methods are remarkably fast in denoising applications, 
segmentation problems require a lot more iterations for those algorithms to converge. 
As stated previously, continuous max-flow is dual with total variation in the contin- 
uous setting under restrictive regularity conditions f28jj . In fact, Nozawa [29] showed 
that there is a duality gap between weighted TV and weighted max-flow under some 
conditions in the continuous domain. Thus, it is important to note that weighted TV 
problems are not equivalent to the weighted maximum flow problem studied in this 
paper. Furthermore, many works assume that the continuous-domain duality holds 
algorithmically, but we show later in this paper that at least in the combinatorial case 
this is not true. 



2 



The previous works for solving the max-flow problem illustrate two difficulties 
with continuous-based formulation, that are (1) the discretization step, which is nec- 
essary for deriving algorithms, but may break continuous-domain properties; and (2) 
the convergence, both of the underlying continuous formulation, and the associated 
algorithm, which itself depend on the discretization. It is very well known that even 
moderately complex systems of PDEs may not converge, and even existence of solu- 
tions is sometimes not obvious [30] . Even when existence and convergence proofs both 
exist, sometimes algorithmic convergence may be slow in practice. For these reasons, 
combinatorial approaches to the maximal flow and related problems are beginning to 
emerge. 

Combinatorial approaches. Discrete calculus [3lJ [32] has been used in recent 
years to produce a combinatorial reformulation of continuous problems onto a graph 
in such a manner that the solution behaves analogously to the continuous formulation 

(e.g., mm)- 

In particular, Lippert presented in [35] a combinatorial formulation of an isotropic 
continuous max-flow problem on a planar lattice, making it possible to obtain a prov- 
ably optimal solution. However, in Lippert's work, parameterization of the capacity 
constraint is tightly coupled to the 4-connected squared grid, and the generalization 
to higher dimensions seems non-trivial. Furthermore it involves the multiplication 
of the number of capacity constraints by the degree of each node, thus increasing 
the dimension of the problem. This formulation did not lead to a fast algorithm. 
The author compared several general solvers, quoting an hour as their best time for 
computing a solution on a 300 x 300 lattice. 

Motivation and contributions. In this paper, we pursue a combinatorial re- 
formulation of the continuous max-flow problem initially formulated by Strang, which 
we term combinatorial continuous maximum flow (CCMF). Viewing our contribution 
differently, we adopt a discretization of continuous max-flow as the primary problem 
of interest, and then we apply fast combinatorial optimization techniques to solve the 
discrctized version. 

Our reformulation of the continuous max-flow problem produces a (divergence- 
free) flow problem defined on an arbitrary graph. Strikingly, CCMF are not equivalent 
to the discretization of continuous max-flows produced by Appleton and Talbot or 
that of Lippert (if for no other reason than the fact that CCMF is defined on an 
arbitrary graph). Moreover, CCMF is not equivalent to the classical max-flow on 
a graph. In particular, we will see that the difference lies in the fact that capacity 
constraints in classical max-flow restrict flows along graph edges while the CCMF 
capacity constraints restrict the total flow passing through the nodes. 

The CCMF problem is convex. We deduce an expression of the dual problem, 
which allows us to employ a primal-dual interior point method to solve it, such as inte- 
rior point methods have been used for graph cuts |36] and second order cone problems 
in general |37j . The CCMF problem has several desirable properties, including: 

1. the solution to CCMF on a 4-connected lattice avoids the metrication errors. 
Therefore, the gridding error problems may be solved without the additional 
memory required to process classical max-flow on a highly- connected graph; 

2. in contrast to continuous max-flow algorithms of Appleton- Talbot (AT-CMF) 
and equivalent, the solution to the CCMF problem can be computed with 
guaranteed convergence in practical time; 

3. the CCMF problem is formulated on an arbitrary graph, which enables the 
algorithm to incorporate nonlocal edges 38, 33, 39 or to apply it to arbitrary 
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clustering problems defined on a graph; and 
4. the algorithm for solving the CCMF problem is fast, easy to implement, 
compatible with multi-resolution grids and is straightforward to parallelize. 
Our computation of the CCMF dual further reveals that duality between total 
variation minimization and maximum flow does not hold for CCMF and combinatorial 
total variation (CTV). The comparison between those two combinatorial problems is 
motivated by several interests: 

1. for clarification: in the continuous domain, the duality between TV and MF 
holds under some regularity constraints. In the discrete anisotropic domain 
(i.e. Graph Cuts), this duality always holds. However, In the isotropic 
weighted discrete case, i.e. in the CTV, AT-CMF or CCMF cases, we are 
not aware of any discretization such that the duality holds. It is not obvious 
to realize that the duality does not hold in the discrete setting even though 
it may in the continuous case. We present clearly the differences between the 
two problems, theoretically, and in term of results; 

2. to expose links between CCMF and CTV: We prove that the weak duality 
holds between the two problems; and 

3. for efficiency: in image segmentation, the fastest known algorithms to opti- 
mize CTV are significantly slower than CCMF. Therefore, there is reason to 
believe that CCMF may be used to efficiently optimize energies for which TV 
has been previously shown to be useful (e.g., image denoising). 

In the next section, we review the formulation of continuous max-flow, derive the 
CCMF formulation and its dual and then provide details of the fast and provably 
convergent algorithm used to solve the new CCMF problem. 

2. Method. Our combinatorial reformulation of the continuous maximum flow 
problem leads to a formulation on a graph which is different from the classical max- 
flow algorithm. Before proceeding to our formulation, we review the continuous max- 
flow and the previous usage of this formulation in computer vision. 

2.1. The continuous max-flow (CMF) problem. First introduced by Iri 
[M] . Strang presents in [15] an expression of the continuous maximum flow problem 



Here we denote by F st the total amount of flow going from the source location s to 
the sink location t, F is the flow, and g is a scalar field representing the local metric 
distortion. The solution to this problem is the exact solution of the geodesic active 
contour (GAC) (or surface) formulation [40]. In order to solve the problem (|2.1|) . the 
Appleton- Talbot algorithm (AT-CMF) [10] solves the following partial differential 
equation system: 



maxF st , 
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Here P is a potential field similar to the excess value in the Push-Relabel maxi- 
mum flow algorithm [?T|. AT-CMF is effectively a simple continuous computational 
fluid dynamics (CFD) simulation with non-linear constraints. It uses a forward finite- 
difference discretization of the above PDE system subject to a Courant-Friedrich-Levy 
(CFL) condition, also seen in early level-sets methods. At convergence, the poten- 
tial function P approximates an indicator function, with values for the background 
labels, and I for the foreground, and the flow F has zero-divergence almost every- 
where. However, there is no guarantee of convergence for this algorithm and, in 
practice, many thousands of iterations can be necessary to achieve a binary P, which 
can be very slow. 

Although Appleton- Talbot is a continuous approach, applying this algorithm to 
image processing involves a discretization step. The capacity constraint ||i^|| < g 
is interpreted as maxFj.^ 2 + maxF y ^ 2 < gf, with F x (i) the outgoing flow of edges 
linked to node i along the x axis, and FyU) the outgoing flow linked to node i along 
the y axis. We notice that the weights are associated with point locations (pixels), 
which will correspond later to a node-weighted graph. 

In the next section, we use the operators of discrete calculus to reformulate the 
continuous max-flow problem on a graph and show the surprising result that the graph 
formulation of the continuous max-flow leads to a different problem from the classical 
max-flow problem on a graph. 

2.2. A discrete calculus formulation. Before establishing the discrete cal- 
culus formulation of the continuous max-flow problem, we specify our notation. A 
graph consists of a pair G = (V,E) with vertices v £ V and edges e <G E C V x V . 
A transport graph G(V, E) comprises two additional nodes, named a source s and a 
sink t, and additional edges linking some nodes to the sink and some to the source. 
Including the source and the sink nodes, the cardinalities of G are given by n = \V\ 
and m = \E\. An edge, e, spanning two vertices, Vi and Vj, is denoted by e^. In this 
paper we deal with weighted graphs that include weights on both the edges and nodes. 
An edge weight is a value assigned to each edge which may be viewed as a capacity 
in the context of a maximum flow problem. The weight of an edge, e^, is denoted 
by gij. In this work, we assume that cjij € R and gij > 0, and use g to denote the 
vector of K m containing the gij for every edge of G. In addition to edge weights, 
we may also assign weights to nodes. The weight of node Vi is denoted by gi. In this 
work, we also assume that gi € R and gi > 0. We use g to denote the vector of R n 
containing the g^ for every edge ey of G. We define a flow through edge e,-j as Fij 
where F^ e R and use the vector F € R m to denote the flows through all edges in 
the graph . Each edge is assumed to be oriented, such that a positive flow on edge 
Bij indicates the direction of flow from Vi to Vj, while a negative flow indicates the 
direction of flow from Vj to Vi ■ 

The incidence matrix is a key operator for defining a combinatorial formulation 
of the continuous max-flow problem. Specifically, the incidence matrix A <E E mxn 
is known to define the discrete calculus analogue of the gradient, while A T is known 
to define the discrete calculus analogue of the divergence (see [32] and the references 
therein). The incidence matrix maps functions on nodes (a scalar field) to functions 
on edges (a vector field) and may be defined as 




(2.3) 
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for every vertex Vk and edge e^-. In our formulation of continuous max- flow, we use 
the expression \A\ to denote the matrix formed by taking the absolute value of each 
entry individually. 

Given these definitions, we now produce a discrete (combinatorial) version of the 
continuous max- flow of (|2.f I) on a transport graph. As in (32J [33] [34] , the continu- 
ous vector field indicating flows may be represented by a vector on the edge set, F. 
Additionally, the combinatorial divergence operator allows us to write the first con- 
straint in (|2.ip as A T F = 0. The second constraint in (|2.1[) involves the comparison 
of the point-wise norm of a vector field with a scalar field. Therefore, we can follow 
[32l [33l [34] to define the point- wise £2 norm of the flow field F as -\/|^4 T |F 2 . In our 
notations here, as in the rest of the paper, F 2 = F ■ F denotes a element-wise prod- 
uct, "•" denoting the Hadamard (element-wise) product between the vectors, and the 
square root of a vector is also here and in the rest of the paper the vector composed 
of the square roots of every elements. 

Putting these pieces together, we obtain 

maxF st , 

s.t. A T F = 0, (2.4) 
\A T \F 2 <g 2 . 

Compare this formulation to the classical max-flow problem on a graph, given in our 
notation as 

maxF st , 

s.t. A T F = 0, (2.5) 
1*1 < 9- 

By comparing these formulations of the traditional max-flow with our combina- 
torial formulation of the continuous max-flow, it is apparent that the key difference 
between the classical formulation and our combinatorial continuous max-flow is in 
the capacity constraints. In both formulations, the flow is defined through edges, but 
in the classical case the capacity constraint restricts flow through an edge while the 
CCMF formulation restricts the amount of flow passing through a node by taking in 
account the neighboring flow values. This contrast-weighting applied to nodes (pix- 
els) has been a feature of several algorithms with a continuous formulation, including 
geodesic active contours [40], CMF [10], TVSeg [16], and [21]. On the other hand, the 
problem defined in (|2.4[) is defined on an arbitrary graph in which contrast weights 
(capacities) are almost always defined on edges. The node-weighted capacities fit 
Strang's formulation of a scalar field of constraints in the continuous max-flow formu- 
lation and therefore our graph formulation of Strang's formulation carries over these 
same capacities. The most important point here is not having expressed the capacity 
on the nodes of the graph but rather how the flow is enforced to be bounded by the 
metric in each node. Bounding the norm of neighboring flow values in each node 
simulates a closer behavior of the continuous setting than if no contribution of neigh- 
boring flow was made as in the standard max-flow problem. Figure [2~T1 illustrates the 
relationship of the edge capacity constraints in the classical max-flow problem to the 
node capacity constraints in the CCMF formulation. The null- divergence constraint 
is also essential in our formulation since it permits us to obtain constant partitions 
almost everywhere in the dual solution introduced in the next section, in particular 
binary ones. 
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Fig. 2.1. The difference between classical max-flow on a graph with the combinatorial contin- 
uous max-flow ( CCMF) on a graph is that classical max-flow uses edge-weighted capacities while 
CCMF uses node-weighted capacities. This difference is manifest in the different solutions obtained 
for both algorithms and the algorithms required to find a solution. Specifically, the solution to the 
CCMF problem on a lattice does not exhibit metrication bias. 



In the context of image segmentation, the vector g varies inversely to the image 
gradient. We propose to use, as in [T5] , 

ff = exp(- j 8||V/|| a ), (2.6) 

where I indicates the image intensity. For simplicity, this weighting function is defined 
for greyscale images, but g could be used to penalize changes in other relevant image 
quantities, such as color or texture. Before addressing the solution of the CCMF 
problem, we consider the dual of the CCMF problem and, in particular, the sense in 
which it represents a minimum cut. Since the cut weights are present on the nodes 
rather than the edges, we must expect the minimum cut formulation to be different 
from the classical minimum cut on a graph. 

2.3. The CCMF dual problem. Classically, the max-flow problem is dual 
to the min-cut problem, allowing a natural geometric interpretation of the objective 
function. In order to provide the same interpretation, we now consider the dual 
problem to the CCMF and show that we optimize a node-weighted minimum cut. 
In the following proposition, we denote the element-wise quotient of two vectors u = 
[ill, ... , life] and v = [vi, . . . ,Vk] by u ■ jv = [m/vi, . . . , itfe/wfc]. We also denote 1" a 
unit vector [1, . . . , 1] of size n, and recall that the square exponent v 2 of a vector v 
represents the resulting vector of the element- wise mulitiplication v ■ v. 

Proposition 2.1. In a transport graph G with m edges, n nodes, we define a vec- 
tor c ofM. m , composed of zeros except for the element corresponding to the source/sink 
edge which is 1 . Let A and v he two vectors of K n . The CCMF problem 

max c T ' F, 

FeR m 

s. t. A T F = 0, (2.7) 
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has for dual 
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\A T \F 2 < g 



X T g 2 + l[l n ./(\A\X)) [{c + A v ) 2 \, (2.8) 



s. t. A > 0, 



equivalently written in (|2.15p . and the optimal solution (F*, X*,v*) verifies 

maxc T F = c T F* = 2X* T g 2 , (2.9) 
and the n following equalities 

\* -\A T \((c + Av*)- /(\A\\*)) =A\*-g 2 . (2.10) 



Proof. The Lagrangian of (|2.7[) is 

F(F, A, = (c T + v T A T )F + A T |A T |F 2 - \ T g\ 

with A € K™ and 1/ £ R" two Lagrange multipliers. We have to find F in the dual 
function is such as VfF(F, A, v) = 0. 

V F L(F, A, i/) = c + Av + 2\A\X ■ F = F = (c + Az/) • /(-2\A\X). (2.11) 

Substituting F in the Lagrangian function , we obtain 

L(A, i/) = (c T +^ T ^l T ) ((c+Ap)-/(-2\A\\)J +X t \A t \ f(c + Av) • /(-2|A|A)) -\ T g 2 . 

The Lagrangian may be simplified as: 

L(X, v) = - (l» ■ /(4|A|A)) T ((c + Avf^j X T g 2 . 

The dual of dUTJ) is also 

nun X T g 2 + \ (l" • /(|A|A)) ((c + Auf^j , (2.12) 
s. t. A > 0. 

Furthermore, the KKT optimality conditions give 

A* ■ (\A T \F* 2 -g 2 \ = 0. (2.13) 

Using the expression of F* from equation (|2.11[) , and substituting it in (|2.13p . we 
obtain 

A* • \A T \({c + Av*) ■ /{\A\X*)\ =4A* ■ g 2 . 



Taking the sum of right hand side and left hand side, 

(c + Av*) 2 ■ /(L4|A*) = 4A* V- (2.14) 

Finally, if we substitute (|2.14p in the dual expression (|2.8p . we have 

max c T F = c T F* = 2X* T g 2 . 
f J 



(a) (b) A (c) v (d) Threshold v at .5 

Fig. 2.2. The dual problem to CCMF is a node-weighted minimum cut in which the variable A 
is a weighted indicator vector labeling boundary nodes and the variable u is a nearly binary vector 
indicating the source/sink regions. As a result, the contours of v are slightly blurry. This is due to 
the equilibrium effect between the two dual variables. In practice, as A is nonzero only in presence 
of a contour, v is binary almost everywhere, except on a very thin line. 

The expression of the CCMF dual may be written in summation form as 

smoothness term source/ sink enforcement 
weighted cut „ ^ 

, 1 { v i~ v jf , l(v s -U t -l) 2 fn , r \ 

rnm£ ^ +1 E + 4 A s + X t ^ 

Vi£V e l3 £E\{s,t} 

s. t. A, > Vi € V. 

Interpretation: The optimal value A* is a weighted indicator of the saturated 
vertices (a vertex Vi is saturated if |j4 t |jF 2 = gf where \A T \i indicates the «th row of 
\A T \): 

AW >l «}f\r=^ (2.16) 
v ' I = otherwise. 

The variables v s and v t are not constrained to be set to and 1, only their difference 
is constrained to be equal to one. Thus their value range between a constant and 
a constant plus one. Let us call this constant 5, and without loss of generality one 
can consider that 6 = 0. The term v is at optimality a weighted indicator of the 
source/sink/saturated vertices partition: 

+ 8 if Vi e S, 

v*{vi) = { a number between (0 + 8) and (1 + 8) if |A T | -F 2 = g(v,i) 2 , 

1 + 8 if Vi € T. 

The expression (|2.9|) of the CCMF dual shows that the problem is equivalent to 
finding a minimum weighted cut defined on the nodes. 

Finally, the "weighted cut" is recovered in (|2.15|) . and the "smoothness term" is 
compatible with large variations of v at the boundary of objects because of a large 
denominator (A) in the contour area. An illustration of optimal A and v on an image 
is shown on Fig. [ 



2.4. Solving the Combinatorial Continuous Max-Flow problem. When 
considering how to optimize the CCMF problem (|2.4[) . the first key observation is 
that the constraints bound a convex feasible region. 



The real valued functions : R m — s- M defined for every node i = 1 , . . . , n as 
fi(F) = lA^F 2 — gf are non- negative quadratic, so convex functions. We define the 
vector f(F) of R» as f(F) = . . . , 

Since the constraints are convex, the CCMF problem may be solved with a fast 
primal dual interior point method (see |42|). which we now review in the specific 
context of CCMF. 

Algorithm 1: Primal dual interior point algorithm 

Data: F i R m = 0, A € (K+)™,^ £ K", /x > l,e > 0. 

Result: F solution to the CCMF problem ()2.7[) such as F st is maximized 

under the divergence free and capacity constraints, and ^, A solution 

to the CCMF dual problem (JUJ). 

repeat 

1. Compute the surrogate duality gap rf = —f(F) T A and set t = /in/rj. 

2. Compute the primal-dual search direction A y such as MA y = r. 

3. Determine a step length s > and set y = y + sA y . (y = [F, A, v] T ) 
until ||r p || 2 < e, ||r d || 2 < e, and rf<e. 



The primal dual interior point (PDIP) algorithm iteratively computes the pri- 
mal F and dual A, v variables so that the Karush-Kuhn- Tucker (KKT) optimality 
conditions are satisfied. This algorithm solves the CCMF problem (|2.7J) by applying 
Newton's method to a sequence of a slightly modified version of the KKT conditions. 
The steps of the PDIP procedure are given in Algorithm [TJ Specifically for CCMF, 
the system MA y = r system may be written 



E"=i AiV 2 /i(F) 


Df(F) T 


A" 




"A F " 




" r d = c + Df(F) T X + Au ' 




-diag(A)D/(F) 


-diag(/(F)) 











r c = -diag(A)/(F) - (1/t) 


(2.17) 


A T 










A, 




r p = A T F 





with fd, r c , and r p representing the dual, central, and primal residuals. Additionally, 
the derivatives are given by 



Vf t (F) = 2\A T \ z -F, Df(F) = 



V/„(F) 3 



, V 2 fi(F) = diag 



2\A T l 



with "•" denoting the Hadamard (element-wise) product between the vectors. 

Consequently, the primary computational burden in the CCMF algorithm is the 
linear system resolution required by (|2.I7|) . In the result section we present execution 
times obtained in our experiments using a conventional CPU implementation. 

Observe that, although this linear system is large, it is very sparse and is straight- 
forward to parallelize on a GPU [331 SUSS], for instance using an iterative GPU solver. 
If it does not necessarily imply a faster solution, the asymptotic complexity of modern 
iterative and modern direct solvers is about the same and, in our experience, there 
has been a strong improvement in the performance of a linear system solve when 
going from a direct solver CPU solution to a conjugate gradient solver GPU solution, 
especially as the systems get larger. 

3. Comparison between CCMF and existing related approaches. 

3.1. Min cut. As detailed in Section 2.3, there is a difference between the CCMF 
and the classic max flow formulation in the capacity constraint. Therefore, the dual 
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problems arc different. As proved by Ford and Fulkcrson [7J, and formalized for 
example in [36) . the min cut problem writes with our notations of section 2.3 



minijr |4u + c|. (3.1) 

u 

We note that the t\ norm of the gradient of the solution u of the above min-cut 
expression turns into an li norm of the gradient of the solution v in the CCMF dual 



3.2. Primal Dual Total variations. Unger et al. [T7J proposed to optimize 
the following primal dual TV formulation 

min max (ui+ij — Uij)Flj + — u i,j)Fi.j + data fidelity, 

(3.2) 

We can note that the CCMF flow capacity constraint in a 4-connected lattice is similar 
to the constraint in (|3.2|) with a slight modification, and would be with finite-element 
notations 



V^ M ) 2 + (^-i) 2 + (^) 2 + (^) 2 < 9i,r (3-3) 

In contrast to this finite element discretization, the provided CCMF formulation of 
continuous max-flow is defined arbitrary graphs. Furthermore, we note that the opti- 
mization procedure employed to solve (|3.2|l generalizes Appleton- Talbot's algorithm. 



3.3. Combinatorial total variations. We now compare the dual CCMF prob- 
lem with the Combinatorial Total Variation (CTV) problem. We note that the CCMF 
dual is different than CTV and discuss the weak duality of the two problems. 

We recall the continuous expression of total variation given by Strang 



min / / g ||Vit|| da; dy, 

JJn (3.4) 
s. t. / uf ds = 1, 



J r 

where £1 represents the image domain and T the boundary of fl. The source and sink 
membership is represented by /, such that f{x,y) > if (x,y) belongs to the sink, 
f(x,y) < if (x, y) belongs to the source, and / = otherwise. Considering a 
transport graph G, and a vector u defined on the nodes, this continuous problem may 
be written with combinatorial operators 



S. t. U s — Ut = 1, 

where the square root operator of a vector v = [v±, . . . , Vk] is an element- wise square 
root operator y/v = [y/vi, • ■ • , \/vk\ ■ Another way to write the same problem is 



m j n X) 9i J Yl ( Ui; ~ u j) 2 ' 

viEV ye y e.E (3.6) 

s. t. u s — ut — 1- 
li 



Wc note that in these equations the capacity g must be defined on the vertices. Al- 
though we describe this energy as "combinatorial total variation" , due to its derivation 
in discrete calculus terms, it is important to note that this formulation is very similar 
to the discretization which has appeared previously in the literature from Gilboa and 
Osher [46], and Chambolle [12] (if we allow g = 1 everywhere). 

3.3.1. CCMF and CTV are not dual. Strang proved that the continuous 
max-flow problem is dual to the total variation problem under some assumptions on 
g. Remarkably, this duality is not observed in the discrete case in which the CCMF 
dual and CTV problems are given by 

minx,, AV + i fl" • /(\A\X)\ f(c + Av) 2 ^ , £ min u g T \/\A^\(Au) 2 , 
s. t. A > 0, s. t. u s — u t = 1. 

We note that the analytic expressions arc different and also not equivalent. The 
duality of the classical max-flow problem with the minimum cut holds because in the 
expression of the Lagrangian function, is possible to deduce a value of A by substituting 
it into the dual problem so that it only depends on v. However, A in the dual CCMF 
problem (|2.9[) depends on several values of neighboring A and v. Thus, the CCMF dual 
problem cannot be simplified by removing a variable, for example by identifying v and 
u in the CTV problem. Said differently, combining a null-divergence constraint with 
an anisotropic capacity constraint in the classical max-flow formulation allows the 
duality to hold; in contrast, in the CCMF formulation, combining a null-divergence 
constraint with an isotropic capacity constraint does not allow such duality to hold. 

Numerically we can also show on examples that the value maxF st of the flow 
optimizing CCMF is not equal to the minimum value of CTV (See Table l3"7Tj) . 

3.3.2. Theoretical links between CCMF and CTV. Even if CCMF is not 
dual with CTV, the two problems are weakly dual. 

In the combinatorial setting the weak duality property is given by 

ll-FH < 9 F T {Au) < g T \\Au\\. (3.7) 

The next property shows that the norm ||F|| = |A T |F 2 verifies weak duality. 

Proposition 3.1. Let G be a transport graph, F a flow in G verifying the 
capacity constraint y^\A T \F 2 < g. Let u be a vector of W 1 (defined on nodes of G). 
Then 

F T Au < g T \J \A T \(Au) 2 . 



Proof. Since we know that y / |^4 T |F 2 < g, the following statement is true 

\j\A T \F 2 \l\AT\{AuY<g T ^J\AT\{Av?. 
We can now show that 



F T Au < y/{\A T \F 2 ) ^\A T \(Au) 2 . 
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Transport graph 
with weights g on 
the nodes 



Combinatorial Continuous 
Max-Flow (CCMF) 



max 

F 



s. t. 



A T F = 0, 
\A T \F 2 < g 2 . 



Combinatorial Total 
Variation (CTV) 



min g T J\AT\{Auf 

U v 

S. t. U s — U t = 1. 



- 

- 1 55 



25 1 1 



; — 2 - 




1.63 
0.408 0.815 : 



0.408 0.408 0.815 
0.47 0.408 

0.87| °' 35 | °' 408 | 



0.76 0.408 




■ — m 



m — M 



0.2 



ctv{u) =E»yz("< -«j) 2 

= 2 + lxV2 + lx\/2 



F at = 1.63 



+ 1 x Vi + o.s 



+2 x ^O.S 2 + 0.2 2 
+2 x 0.2 
= 8.16. 



Table 3.1 

Example illustrating the difference between optimal solutions of combinatorial continuous max- 
imum flow problem (CCMF), and combinatorial total variation (CTV). 



Using summation notation, then by the Cauchy-Schwartz inequality, 



E E F ij( U i- U i) ^ E 



i£Ve i:i eE 

We conclude that 



l£V 



. e v E 



F Au < J(\AT\F*) J\AT\(Au) 2 < g \/ \ A T \(Au) 2 



In terms of energy value, this proposition means that the CCMF energy, that is 
to say the flow, is always smaller than the combinatorial total variation. 

Proposition 3.2. Let F be a compatible flow verifying the constraints in 
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and u a vector of K™ such as u s — u t = 1 . Then, 

F st <g T ^AT{Auf. (3.8) 

Proof. In the combinatorial setting, the Green formula gives 

F T Au = u T A T F. 
Let a be a vector of W 1 defined for each node Vi as 

!— 1 if Vi belongs to the sink, 
1 if Vi belongs to the source, (3.9) 
otherwise. 

We can provide the following equivalent formulation of the CCMF problem (|2.4[) using 
an incidence matrix A of the transport graph deprived of the sink-source edge: 

maxF st , 
s.t. A T F = F st a, 
\A T \F 2 <g. 

As F verifies the divergence free constraint A T F = F st a, 

u T A T F = u T F st a, 

and as the equation u s — Ut = 1 may be written equivalently a T u = 1, we conclude 
that 

u T F st a = F st , 
and by weak duality (Property 13. II) . 

F st < g T \J A T (Au) 2 . 

□ 

This property should be of interest for extending CCMF to applications other 
than segmentation, which is outside the scope of this paper. In the next section, we 
present the performance of our approach in the context of segmentation. 

4. Results. We now present applications of Combinatorial Continuous Maxi- 
mum Flow in image segmentation. To be used as a segmentation algorithm, three 
solutions are possible: the dual variable v may be used directly if the user needs a 
matted result, otherwise v may be thrcsholdcd, or finally an isolinc or isosurface may 
be extracted from v. 

In the introduction we discussed how several works are related to CMF. Some are 
equivalent or slight modifications of AT-CMF for non-segmentation applications [16j 
[TTl [TH [19] . As in the original AT-CMF work, none of these come with a convergence 
proof. In contrast, the work of |47j and |35j are provably convergent but both are 
very slow, and do not generalize easily to 3D image segmentation. Consequently, it 
seems reasonable to compare our segmentation method to the original AT-CMF as 
representative of the continuous approach, as well as graph cuts, which represent the 
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GO 



CCMF 



GO 



CCMF 



Fig. 4.1. Brain segmentation, (a) Original image with foreground and background seeds. (b,c,d) 
Segmentation obtained with (b) graph cuts (GC), (c) AT-CMF, threshold of P (obtained after 10000 
iterations), (d) CCMF, threshold of v (15 iterations). 



purely discrete case. Finally, even if the total variation problem is different than the 
CMF problem (existence of a duality gap in the continuous [29]), the two problems 
are related enough to be compared. Specifically, we will also compare with Chambolle 
and Pock's recent work [21j which presented an algorithm for optimizing a TV-based 
energy for image segmentation. 

Our validation is intended to establish three properties of the CCMF algorithm. 
First, we establish that the CCMF does avoid the metrication artifacts exhibited 
by conventional graph cuts (on a 4-connected lattice). This property is established 
by examples on a natural image and the recovery of the classical catenoid structure 
as the minimal surface spanning two rings. Second, we compare the convergence of 
the CCMF algorithm to the AT-CMF algorithm to show that the CCMF algorithm 
converges more quickly and in a more stable fashion. Finally, we establish that our 
formulation of the CMF problem does not degrade segmentation performance on a 
standard database. In fact, because of the reduction in metrication error our algorithm 
gives improved numerical results. For this experiment, we use the GrabCut database 
to compare the quality of the segmentation algorithms. In addition to the above tests, 
we demonstrate through examples that the CCMF algorithm is also flexible enough 
to incorporate prior (unary) terms and to operate in 3D. 

4.1. Metrication artifacts and minimal surfaces. We begin by comparing 
the CCMF segmentation result with the classical max- flow algorithm (graph cuts). 
Figure I4TT1 shows the segmentation of a brain, in which the contours obtained by graph 
cuts are noticeably blocky in the areas of weak gradient, while the contours obtained 
by both AT-CMF and CCMF are smooth. 

In the continuous setting, the maximum flow computed in a 3D volume produces 
a minimal surface. The CCMF formulation may be also recognized as a minimal 
surface problem. In the dual formulation, the objective function is equivalent to a 
weighted sum of surface nodes. In [10], Appleton and Talbot compared the surfaces 
obtained from their algorithm with the analytic solution of the catenoid problem to 
demonstrate that their algorithm was a good approximation of the continuous minimal 
surface and was not creating discretization artifacts. The catenoid problem arises from 
consideration of two circles with equal radius whose centers lie along the z axis. The 
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Fig. 4.2. The catenoid test problem: The source is constituted by two full circles and sink by the 
remaining boundary of the image, (a) Surface computed analytically, (b) isosurface of P obtained 
by CMF, (c) isosurface of v obtained by CCMF. The root mean square error (RSME) has been 
computed to evaluate the precision of the results to the surface computed analytically. The RSME 
for CMF is 1.98 and for CCMF 0.75. The difference between those results is due to the fact that 
the CMF algorithm enforces exactly the source and sink points, leading to discretization around the 
disks. In contrast, the boundary localized around the seeds of v is smooth, composed of grey levels. 
Thus the resulting isosurface computed by CCMF is more precise. 

minimal surface which forms to connect the two circles is known as a catenoid. The 
catenoid appears in nature, for example by creating a soap bubble between two rings. 
In order to demonstrate that CCMF is also finding a minimal surface, we performed 
the same catenoid experiment as was in [IU] . The results are displayed in Figure 14.21 
where we show that CCMF approximates the analytical solution of the catenoid with 
even greater fidelity than the AT-CMF example. 

In order to verify that the solution obtained by CCMF approximates perimeters 
of planar objects in 2D, we segmented several shapes - squares, discs - with known 
perimeters, scaled at different size, and compared the analytic perimeters with the 
energies obtained by CCMF. The results are reported in Fig. 14.31 where we observe 
that the true boundary length is proportional to the energy obtained with CCMF. 

4.2. Stability, convergence and speed. We may compare the segmentation 
results using v to Applcton- Talbot's result using P. We recall that AT-CMF solves the 
partial differential equation system (|2 .3[) in order to solve the continuous maximum 
flow problem (|2.ip . but no proof was given for convergence. The potential function 
P approximates an indicator function, with values for the background labels, and 1 
for the foreground. It can be difficult to know when to stop the AT-CMF algorithm, 
since the iterations used to solve the AT-CMF algorithm may oscillate, as displayed 
in Figure 14.41 on a synthetic image. In contrast, the CCMF algorithm is guaranteed 
to converge and smoothly approaches the optimum solution. 

We also compare segmentation results with results obtained by the recent algo- 
rithm of Chambollc and Pock [21] optimizing a total variation based energy using a 
dual variable. In comparison to our work, the dual variable is not a flow (no constraint 
on the divergence and no proof of convergence toward a minimal surface as defined 
in [15j or [10j). In the literature, the continuous-domain duality between TV and 
max- flow is assumed, but in computational practice they are really different. Nev- 
ertheless, even as the method of Chambolle-Pock is solving a different problem, we 
performed numerical comparative tests. The problem of Chambolle and Pock (now 
denoted CP-TV) for binary segmentation optimizes 

min max F T (Au) + g T u+ hard constraints attachment, (4-1) 
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Fig. 4.3. Comparison of perimeters computed analytically and with Graph cuts and CCMF. 
To estimate the boundary length with CCMF, segmentations have been produced using the input 
images in (b) with the foreground and background seeds that appears in (a), (c) : plot showing 
the linear relation between the true perimeters and the energies obtained with CCMF. The symbols 
used to represent the dots correspond to the objects segmented in the images. A similar relation 
of proportionality is obtained with CP-TV. (d) : value of the cut obtained for a diagonal and a 
vertical line of same length using graph cuts. The difference observed explains (e) : plot showing 
the nonlinear relation between the true perimeters and the energies obtained with Graph cuts. 



where A £ R mx ™ j s the incidence matrix of the graph of n nodes and m edges defined 
on the image, g £ R™ is the metric on the nodes, defined in equation (|2.6p . 

The CCMF algorithm is faster than both AT-CMF and CP-TV for 2D image 
segmentation. We have implemented CCMF in Matlab. We used an implementa- 
tion of Applcton- Talbot's algorithm in C++ provided by the authors, and a Matlab 
software implementing CP-TV on CPU, also provided by the respective authors. The 
three implementations make use of multi-threaded parallclization, and the CPU times 
reported here were computed on a Intel Core 2 Duo (CPU 3.00GHz) processor, with 
2 Gb of RAM. The CCMF average computation time on a 321 x 481 image is 181 
seconds after 21 iterations. For AT-CMF, 80000 iterations require 547 seconds. For 
CP-TV, the average number of iterations needed for convergence is 36000, requiring 
an average time of 1961 seconds on CPU, and the median computation time is 1406 
seconds. Note that a GPU implementation of CP-TV, also provided by the authors, 
achieves a speed gain factor of about 100 on a NVidia Quadro 3700. Although it 
was not pursued here, it should be noted that the CCMF optimization approach 
could also fully benefit from parallclization (e.g., on a GPU) because the core com- 
putation is to solve a linear system of equations. Realizing the benefit of a GPU 
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solution would require the use of an iterative algorithm such as conjugate gradients 
or multigrid [331 H3j , which would make comparisons to the direct solver used here 
less immediate. However, previous examples in the literature have suggested that this 
change in solver has not created difficulties in order to realize the benefits of a GPU 
architecture [45] . 

Sometimes, it may not always be necessary to wait so long for the complete 
convergence of CCMF (or AT-CMF) to obtain acceptable results. In cases where 
images exhibiting sufficiently strong gradients, one iteration may be enough to obtain a 
satisfying segmentation. On such an image, one iteration of CCMF, and 100 iterations 
of AT-CMF show acceptable approximate results reached only after about 2 seconds 
for either algorithm. However, one cannot always rely on strong edges, and the power 
of our CCMF formulation is to behave in a predictable fashion even when edges are 
weak. 

We may also compare the computation time of CCMF to the optimization of total 
variation using the Split Bregman method [26]. Optimizing TV on a 100 x 100 image 
requires 5000 iterations and takes 23 seconds with a sequential implementation of 
Split Bregman in C. On the same image, CCMF required only 17 iterations to reach 
the convergence criteria ||r<i|| < 1 and \\rj\\ < 2, taking 4.7 seconds with a Matlab 
implementation (using a 2-threaded solver). We conclude that TV-based methods 
appear to be quite slow in the context of image segmentation, although we employed 
the Split Bregman algorithm which is known as one of the fastest algorithms for 
TV optimization for image denoising, and the very recent CP-TV algorithm. This 
difference in speed between the denoising and segmentation applications is due to the 
very large number of iterations required to convergence to a binary segmentation. 




Fig. 4.4. Segmentation of an artificial image with AT-CMF (top row) and CCMF (bottom 
row). Top row (AT-CMF): (a) Image where the black and white discs are seeds. AT-CMF result 
stopped in (b) after 100 iterations, (c) 1000 iterations, (d) 10000 iterations. Bottom row (CCMF): 
(e) Image where the black and white discs are seeds, CCMF result v after (f) 1 iterations, (g) v 
after 15 iterations iterations and (h) threshold of the final v. 
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Dice Cocff. 


GC 


AT-CMF 


CP-TV 


CCMF 




mean 


95.2 


94.9 


95.2 


95.3 


First set of seeds 


median 


96.2 


96.1 


96.3 


96.3 




stand, dev. 


4.3 


4.1 


4.1 


4.1 




mean 


89.3 


89.7 


89.1 


89.5 


Second set of seeds 


median 


91.2 


92.0 


91.7 


92.3 




stand, dev. 


8.4 


7.7 


8.5 


8.6 



Table 4.1 

Dice coefficient (percentage) computed between the segmentation mask and the ground truth 
image (provided by GrabCut database) . The lines above the double bar show results for the first set 
of seeds, and the lines below the double bar show results obtained with the second set of seeds. 



4.3. Segmentation quality. In this experiment, we compared our CCMF for- 
mulation to the AT-CMF formulation and conventional graph cuts on the problem 
of image segmentation, to determine if there were strong performance differences be- 
tween the formulations. We expect that there would not be, since in principle all three 
formulations are trying to "minimize the cut" , but have slightly different definitions 
of the cut length. The primary advantage that we expect from our formulation is in 
the reduction of metrication artifacts (as compared to conventional graph cuts) and 
speed/convergence (as compared to AT-CMF). 

Our experiment consists of testing the Graph Cut, AT-CMF, CP-TV, and Com- 
binatorial Continuous Max-Flow algorithms on a database with the same seeds. We 
used the Microsoft 'GrabCut' database available online [35]: which is composed of fifty 
images provided with seeds. From the seed images provided, it is possible to extract 
two different possible markers for the background seed. Examples of such seeds are 



shown in Fig. 4.5(a) Different convergence criteria are available for CCMF, such as 
the duality gap and norms of the residuals. However, for the CMF algorithm, we do 
not have any satisfying criteria. Bounding the number of non binary occurrences of 
P does not mean that the convergence is reached, because of possible oscillations. 
An intermediate result after ten thousand iterations may be significantly different 
from the result reached after one hundred thousand. Consequently, we have run the 
AT-CMF algorithm until we were convinced to have reached convergence, i.e. when 
P was nearly binary and did not change significantly when we doubled the number 
of iterations. For half of the images in the GrabCut database, AT-CMF algorithm 
required more than 20000 iterations to reach convergence, for a third of the images, 
more than 80000 iterations, and for 1/4 of the images, more than 160000 iterations. 
Binary convergence was still not reached after even 500000 iterations for the rest of 
the images (1/4), but we stopped the computation anyway. The Chambolle-Pock 
TV based algorithm for binary segmentation is provably convergent and benefits also 
from several possible convergence criteria. The convergence criteria used is a ratio 
of changed pixels from two successive intervals of iterations being smaller than an 
epsilon that we fixed to 10~ 7 in order to obtain satisfying results. In practice, 1/4 
of the images converged in less than 60000 iterations and for the rest of them we 
increased the maximum number of iterations up to 200000. In contrast, CCMF only 
needed 21 iterations on average, and never more than 27, to reach the convergence 
criteria \\rd\\ < 1 and \\rj\\ < 2. We noted that for all methods, segmentation quality 
degraded quickly if these conditions were not respected. 

Figure |4~31 displays the performance results for these algorithms. The Dice coeffi- 
cient is a similarity measure between sets (segmentation and ground truth), ranging 
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Fig. 4.5. (a) : Seed images belonging to the first (top) and second (bottom) set of seeds provided 
by the GrabCut database, (b) Segmentations by Graph Cuts computed with seeds from the two sets, 
(c) Segmentations by CCMF. 

from to 1.0 for no match and a perfect match respectively. All the tested algorithms 
show very good results, with a Dice coefficient of 0.95-0.97 for the well positioned 
seeds, and 0.89-0.92 for the second set of seeds, further away from the objects. The 
CCMF and AT-CMF results are really close, and the mean is better than the GC and 
the CP-TV results. 

4.4. Extensions. The primary focus of our experiments were to demonstrate 
that our CCMF algorithm achieves a solution which does not exhibit metrication 
artifacts, that it is fast with provable convergence and that the segmentation quality is 
high. In the remainder of this section, we show that the algorithm can also incorporate 
unary terms and be equally applied in 3D. In fact, since CCMF is defined for an 
arbitrary graph, it could even be used to perform clustering in this more generalized 
framework. However, the benefit of avoiding gridding artifacts is less meaningful when 
performing clustering on an arbitrary graph, and therefore we omit any examples of 
this nature. 

4.4.1. Unary terms. A simple modification of the transport graph G permits 
the use of unary terms to automatically specify objects to be segmented. This ap- 
proach is similar to the use of unary terms in the graph cuts computation [5] . The 
placement of unary terms for adding data attachment constraints in the max-flow 
problem is performed by adding edges linking every node of the lattice to the source 
and to the sink. In the case of the CCMF problem, as weights arc defined on the nodes, 
we need to add intermediary nodes between the grid and source on the one hand, and 
the grid and sink on the other. However, since these intermediary nodes have just 
two edges incident on them, these node weights are equivalent to edge weights, mean- 
ing that our construction is equivalent to the use of unary terms for AT-CMF that 
was pursued by Unger et al. |17j . In our case, we used weighted intermediary nodes 
simply to keep the CCMF framework consistent. Considering that the original lattice 
is composed of n nodes, we add for each node an "upper" node linked to the source 
and a "lower" node linked to the sink. An example of this construction is shown in 
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Fig.SH 





Fig. 4.6. Example of graph construction to incorporate unary terms for unsupervised segmen- 
tation. Each node is connected to a source node and sink node through an intermediary node which 
is weighted to reflect the strength of the positive and negative unary term. This construction is then 
applied with a simple appearance prior to automatically segment two images. 

The weights of the additional nodes may be set to reflect the node priors for a 
particular application. For image segmentation, given mean background and fore- 
ground color BC and FC, we can set the capacities of the background nodes to 
gt = cxp(—f3(BCi — Ii) 2 ) and foreground nodes to g± = cxp(—/3(FCi — li) 2 )- Exam- 
ples of results using these appearance priors are shown on Fig. 14.61 

4.4.2. Classification. As the general CCMF formulation is defined on arbi- 
trary graphs, it is possible to employ it in tasks beyond image segmentation. For 
instance CCMF may be employed to solve general problems of graph clustering, even 
on networks with no meaningful embedding such as social networks. We show a pos- 
sible application in a classification example, considering the social network studied 
by Zachary [49]. After the split of a university karate club due to a conflict between 
its two leaders, Zachary's goal was to study if it was possible to predict the side 
joined by the members, based only on the social structure of the club. Classically, 
the graph is built by associating each member to a node, and edges link two members 
when special affinities are known between them. As weights are associated here to the 
strength of coupling between members, different strategies are possible for assigning 
the weights onto the nodes, which is necessary for using CCMF. For example, a given 
node/member may be assigned by the mean of affinity with its neighboring members 
in the graph. This weighting strategy works well in this example (See Fig. 14. 7\ . An- 
other strategy would be to compute the solution in the dual graph, that is in the graph 
where nodes are replaced by edges and (weighted) edges are replaced by (weighted) 
nodes. We have shown in an example the ability of CCMF for classification, a task 
that can not be treated with finite element-based methods such as AT-CMF. Evalu- 
ating the performances of CCMF for classification may be a topic for future research. 



4.4.3. 3D segmentation. For 3D image segmentation, the minimal surface 
properties of CCMF generate good quality results, as shown in Fig 14.81 The CCMF 
formulation applies equally well in 2D or 3D, since CCMF is formulated on an arbi- 
trary graph (which may be a 2D lattice, a 3D lattice or an even more abstract graph). 
In 3D, our CCMF implementation is suffering from memory limitations in the direct 

21 




(a) Network and true membership after split (b) Classification using CCMF 



Fig. 4.7. Zachary's karate club network. The two leaders in conflict are represented by the 
left hand side and right hand side nodes. Just one node is misclassified with CCMF. However, this 
node is known classically to be an unusual situation within the social network (see \49j ) which is 
not captured by any known algorithm. 

solver we used, limiting its performances. Future work will address this issue using a 
dedicated solver. 

5. Conclusion. In this paper, we have presented a new combinatorial formula- 
tion of the continuous maximum flow problem and a solution using an interior point 
convex primal-dual algorithm. This formulation allows us to optimize the maximum 
flow problem as well as its dual for which we provide an interpretation as a minimal 
surface problem. This new combinatorial expression of continuous max-flow avoids 
blockiness artifacts compared to graph cuts. Furthermore, the formulation of CMF 
on a graph reveals that it is actually the fact that the capacity constraints are ap- 
plied to point-wise norms of flow through nodes that allows us to avoid metrication 
errors, as opposed to the conventional graph cut capacity constraint through individ- 
ual edges. Additionally, it was shown that our CCMF formulation provides a better 
approximation to the analytic catenoid than the conventional AT-CMF discretization 
of the continuous max-flow problem. Contrary to graph cuts, we showed that CCMF 
estimates the true boundary length of objects. Finally, unlike finite-element based 
methods such as AT-CMF, the CCMF formulation is expressed for arbitrary graphs, 
and thus can be employed in a large variety of tasks such as classification. 

We provide in this paper an exact analytic expression of the dual problem, con- 
vergence of the algorithm being guaranteed by the convexity of the problem. In terms 
of speed, when an approximate solution is sufficient, our implementation of CCMF 
in Matlab is competitive to the Appleton- Talbot approach, which uses a system of 
PDEs, and a C++ implementation. The Appleton- Talbot algorithm has the significant 
drawback of not providing a criterion for convergence. In practice, this translates to 
long computation times when convergence is difficult for the AT-CMF algorithm. In 
contrast, our algorithm in this case is much faster, as we observe that the number 
of iterations required for convergence does not vary much for CCMF (less than a 
factor of two). The CCMF algorithm is simple to implement, and may be applied to 
arbitrary graphs. Furthermore, it is straightforward to add unary terms to perform 
unsupervised segmentation. 

We have also compared our algorithm to known weighted combinatorial TV min- 
imization methods (CTV optimized with split-Bregman, CP-TV). We have shown 
that our results are generally better for segmentation than combinatorial TV-based 
methods, and that our implementation on CPU is much faster and more predictable. 
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Specifically, the number of iterations required for convergence does not vary much for 
CCMF, whereas it can vary of more than a factor of ten for CP-TV. The deep study 
of the relationships between CCMF and combinatorial TV reveals that, in contrast 
with expectations from their duality under restrictive assumptions in the continuous 
domain, their duality relationship is only weak in the combinatorial setting. In fact 
max-flow and total variation are different problems with different constraints, yield- 
ing different algorithms and different results. One key difference between max-flow 
and total variation is that max-flow algorithms were developed as segmentation al- 
gorithms, following the graph cuts framework. One consequence is that max-flow 
formulations have a null divergence objective, which is not present in total variation 
formulations. Null divergence can be viewed as a consequence or a necessity in or- 
der to obtain constant partitions almost everywhere, in particular binary ones. This 
difference is important because in the proposed CCMF framework we impose a tight 
null divergence constraint, which to our knowledge is novel for an isotropic formula- 
tion (graph-cuts have always had this constraint). AT-CMF and similar frameworks 
achieve null divergence if and when convergence is achieved. There is no null di- 
vergence constraint at all in total variation frameworks. As a consequence, while in 
practice it is possible to compare total variation and max flow formulations, they 
should be treated differently. However, the strong computational performance and 
segmentation quality results of CCMF as compared to TV (using the strongest known 
optimizations methods) suggests that it may be advantageous to apply our CCMF 
formulation to problems for which TV has proven effective (such as filtering). 

Several further optimizations of CCMFs are possible, for instance using multi- 
grid implementations, the possibility to use GPU to solve the iterative linear system, 
the use of a dedicated solver for the particular sparse linear system involved in the 
computation. We also plan to compare the efficiency obtained with the interior point 
method to a first order algorithm for solving the CCMF problem. 

Ultimately, we hope to employ combinatorial continuous maximum flow as a pow- 
erful segmentation algorithm which avoids metrication artifacts and provides a fast 
solution with provable convergence. Furthermore, we intend to explore the potential 
of CCMF to optimize other energy functions for which graph cuts or total variation 
have proved useful, such as surface reconstruction or efficient convex filtering. 
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